Guaranteed Non-Orthogonal Tensor Decomposition via Alternating Rank-1 Updates

In this paper, we provide local and global convergence guarantees for recovering CP (Candecomp/Parafac) tensor decomposition. The main step of the proposed algorithm is a simple alternating rank-1 update which is the alternating version of the tensor power iteration adapted for asymmetric tensors. Local convergence guarantees are established for third order tensors of rank k in d dimensions, when k = o ( d ) and the tensor components are incoherent. Thus, we can recover overcomplete tensor decomposition where the tensor rank k is larger than the dimension d. We also strengthen the results to global convergence guarantees under stricter rank condition k ≤ βd (for arbitrary constant β > 1) through a simple initialization procedure where the algorithm is initialized by top singular vectors of random tensor slices. Furthermore, the approximate local convergence guarantees for p-th order tensors are also provided under rank condition k = o ( d ) . The guarantees also include tight perturbation analysis given noisy tensor.